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I  ntroduction 


This  report  will  review  progress  in  the  area  of  human  performance  modeling  within 
the  context  of  a  Human  Systems  Integration  (HSI)  approach,  with  particular  focus  on 
how  aspects  of  personality  could  contribute  to  human  performance  modeling.  While 
personality  has  been  demonstrated  to  be  a  modest  predictor  of  job  performance  (Barrick 
&  Mount,  1991;  McHenry,  Hough,  Toquam,  Hanson,  &  Ashworth,  1990)  the  relationship 
between  personality  and  discrete  task  performance  in  varying  environmental  conditions 
has  not  been  rigorously  studied.  This  report  will  detail  how  personality  constructs 
originally  developed  for  purposes  of  selection  and  classification  might  be  utilized  to 
enhance  current  models  of  human  performance  and  how  personality  constructs  could  be 
included  in  a  specific  Human  Performance  Modeling  (HPM)  tool,  the  Improved 
Performance  Research  Integration  Tool  (IMPRINT).  Other  areas  where  additional  work 
could  contribute  to  a  broader,  more  useful  HSI  process  for  the  Navy  will  also  be 
addressed. 

Recent  trends  have  resulted  in  the  recognition  that  construction  of  accurate  models 
of  human  behavior  will  benefit  the  Navy  in  myriad  ways.  As  noted  by  Zachary, 

Campbell,  Laughery,  Glenn,  and  Cannon-Bowers  (2001): 

“At  the  same  time,  both  industry  and  the  government  (particularly  the 
military)  are  facing  challenges  that  are  driving  the  need  for  human 
performance  models.  For  example,  the  Department  of  Defense  (DoD) 
is  requiring  that  future  Navy  ships  must  be  built  with  reduced  budgets 
and  operated  by  vastly  smaller  crews,  and  must  operate  effectively  and 
efficiently  in  mission  environments  that  are  complex,  difficult  to 
define,  and  rapidly  changing.  Thus,  the  human  component  is,  more 
than  ever,  the  critical  component  to  mission  success.” 

The  Defense  Modeling  and  Simulation  Office  (DMSO)  “identified  the  capability  to 
robustly  represent  individual  and  group  behaviors  as  a  critical  need”  (Gery,  Doyal,  Brett, 
Lebiere,  Biefield,  &  Martin,  2003).  While  not  addressing  human  behavior  models 
directly,  the  U.S.  Government  Accountability  Office1  (GAO,  2003)  recommended  the 
Navy  “require  that  ship  programs  use  human  systems  integration  to  establish  crew  size 
goals  and  help  achieve  them”  and  “formally  establish  a  process  to  examine  and  facilitate 
the  adoption  of  labor  saving  technologies. ..across  Navy  systems.”  Implementation  of 
these  recommendations  will  require  prediction  of  performance  outcomes  of  man- 
machine  systems  through  the  use  of  human  behavior  models. 


Benefits  of  Adding  a  Personality  Construct  Performance  Moderator 

Adding  a  personality  construct  as  a  performance  moderator  to  the  current  battery  of 
environmental  stressors,  training,  and  personnel  moderators  would  enhance 
IMPRINT’S  ability  to  simulate  human  behavior  and  define  the  complete  Warfighter- 


1  Known  as  the  Government  Accounting  Office  prior  to  July  7,  2004. 
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system.  The  most  immediate  result  of  implementing  a  personality  construct 
performance  moderator  would  allow  the  analysis  of  personality  influences  on 
Warfighter-system  performance  from  mission  to  function  and  to  task.  Additionally, 
users  could  identify  the  types  of  personality  necessary  to  enhance  overall  mission 
effectiveness.  This  has  the  potential  to  directly  support  the  personnel  selection  process 
throughout  the  acquisition  cycle.  Personnel  domain  professionals  could  use  the  model 
data  to  refine  recruitment  and  retention  throughout  the  lifecycle  of  the  system,  ensuring 
consistent  mission  performance. 


Human  Systems  I  ntegration 


Human  Systems  Integration  is  a  means  to  reduce  overall  life  cycle  costs  through 
consideration  of  both  the  system  and  the  human  component  at  an  early  stage  in  the 
acquisition  of  new  systems.  As  noted  in  Figure  l  below  (GAO,  2003),  a  significant 
portion  of  system  life  cycle  costs  are  determined  early  in  system  development. 


Milestone  decisions  made  here.. 


lock  in  80  -  90  percent  of  costs  here.. 


]  Development  cost 

Procurement  cost 
Operating  and  support  o 


System  Me  cycle 


Source:  U  S.  Navy. 


Figure  1.  System  life  cycle  costs. 


HSI  is  a  methodology  for  ensuring  that  systems  design  and  development 
incorporates  the  integration  of  human  performance  attributes,  including  cognitive, 
psychomotor,  and  perceptual  attributes,  with  system  attributes,  including  hardware  and 
software,  to  minimize  Total  Ownership  Cost  (TOC).  The  primary  focus  of  HSI  is  on  the 
development  of  a  system  with  the  human  as  an  integral  component,  not  an  afterthought. 
However,  in  order  to  have  a  robust  HSI  process  it  is  necessary  to  construct  robust 
models  of  human  behavior.  Models  and  simulations  that  include  human  behavior  allow 
inexpensive  exploration  of  concepts  early  in  system  development.  These  concepts  can  be 
analyzed  and  included  in  the  system  design  trade-space  throughout  the  development  of 
the  system. 
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Human  Behavior  Representation 

Human  behavior  representation  (HBR)  denotes,  “...a  computer-based  model  that 
mimics  either  the  behavior  of  a  single  human  or  the  collective  action  of  a  team  of 
humans”  (Pew  &  Mavor,  1998).  The  phrase  human  performance  modeling  is  also  used 
interchangeably  with  human  behavior  representation,  although  human  performance 
modeling  may  also  be  used  more  globally  (i.e.,  HBR  can  be  a  subset  of  HPM).  More 
generally,  HBR  is  a  group  of  modeling  techniques  designed  to  replicate  the  actions  or 
behavior  of  people.  Generally  these  techniques  instantiate  the  relevant  behavioral  model 
as  software  code  embedded  in  a  computer  simulation  that  allows  for  analysis  of 
inputs/outputs  and  internal  processes  over  time.  These  models  may  focus  on  particular 
aspects  of  human  behavior  such  as  decision  making  or  they  may  be  linked  with  other 
models,  such  as  perceptual  processes,  that  consider  more  comprehensive,  integrated 
aspects  of  human  behavior.  There  have  been  few  efforts  that  have  attempted  to 
instantiate  aspects  of  personality  into  computer  models  of  behavior.  Some  of  these  will 
be  discussed  in  detail  in  a  later  section. 

Personality  constructs  that  have  been  developed  for  selection  and  classification  are 
customarily  not  used  to  predict  discrete  task  performance  at  the  granularity  that  most 
instantiated  human  performance  models  are  able  to  predict.  The  interest  for  selection 
and  classification  has  been  in  global  measures  of  job  performance.  According  to 
Campbell,  McHenry,  and  Wise  (1990),  whose  definition  of  job  performance  has  proven 
of  great  value  to  the  field  of  personnel  psychology,  job  performance  focuses  on 
observable  behavior  in  support  of  organizational  goals.  They  define  job  performance  as: 


“...observable  things  people  do  (i.e.,  behaviors)  that  are  relevant  for 
the  goals  of  the  organization.  The  behaviors  that  constitute 
performance  can  be  scaled  in  terms  of  the  level  of  performance  they 
rep  resent.... There  is  not  one  outcome,  one  factor,  or  one  anything  that 
can  be  pointed  to  and  labeled  as  job  performance.  Job  performance 
really  is  multidimensional.”  (pg.  314) 

The  particular  aspect  of  HBR  that  we  were  interested  in  for  the  Navy  HSI  initiative2 
was  the  modeling  of  a  component  of  job  performance,  namely  task  performance.  And  we 
were  particularly  interested  in  task  performance  as  characterized  by  two  performance 
outcomes,  task-time-to-complete  and  task  accuracy.  IMPRINT  provides  data  both  on 
task-time-to-complete  and  on  task  accuracy.  IMPRINT  stressor  algorithms  currently 
built  into  the  tool  impact  performance  outcomes  based  on  these  measures.  Task 
performance  continues  to  more  clearly  define  the  construct  of  job  performance  from  a 
behavioral  perspective.  Task  performance  and  contextual  or  organizational  citizenship 
behaviors  form  two  focal  domains. 


2  The  Navy  HSI  initiative  was  originally  entitled  Systems  Engineering  Acquisition  and  Personnel 
Integration,  or  SEAPRINT,  but  was  later  renamed  for  clarity. 
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Task  Performance 


Task  performance  can  be  categorized  into  two  types.  The  first  type  “...consists  of 
activities  that  transform  raw  materials  into  the  goods  and  services  that  are  the 
organization’s  products”.  The  second  type  “...consists  of  activities  that  service  and 
maintain  the  technical  core...”  (Motowidlo,  Borman,  &  Schmit,  1997,  pg  75).  The  vast 
majority  of  HBR  efforts  within  the  HSI  context  are  investigations  of  differential 
performance  outcomes  involving  task  performance  of  the  first  type.  The  interest  is  in 
whether  or  not  job  tasks  can  be  clearly  expressed  and  ordered  so  that  predictions  can  be 
made  about  the  ability  of  a  proposed  man-machine  configuration  to  meet  desired 
operational  performance  requirements.  Task  models  used  in  these  predictive  efforts 
generally  ignore  the  context  within  which  the  tasks  are  actually  performed.  Generally, 
the  assumption  is  that  the  human  component  performs  the  necessary  tasks  much  as  an 
automaton,  not  requiring  breaks  or  feeling  task  overload  stress  and  the  focus  is  on  task 
performance. 


Performance  Moderators 

Performance  moderators  are  variables  that  impact  performance  behavior.  These 
moderators  can  be  important  contributors  to  human  performance  models  for  various 
reasons.  For  our  consideration,  the  most  important  reason  is  that  these  moderators  can 
change  performance  outcomes.  Pew  and  Mavor  (1998)  distinguish  between  moderators 
as  being  external  or  internal. 


External  Moderators 

External  moderators  are  external  to  the  individual.  They  are  generally  some  type  of 
physical  stressor  and  include  heat,  noise,  vibration,  and  physical  workload.  Many  of 
these  stressors  have  empirical  studies  linking  stressor  levels  with  clear  performance 
degradation.  IMPRINT  incorporates  various  algorithms  that  allow  modelers  to  account 
for  the  impact  of  physical  stressors  on  task  performance,  leading  to  differential  mission 
performance  outcomes  (Mitchell,  2000). 


I  nternal  Moderators 

Internal  moderators  are  internal  to  the  individual.  Internal  moderators  include 
intelligence,  level  of  expertise,  emotions,  and  personality.  In  IMPRINT  differential 
performance  outcomes  can  be  produced  by  manipulating  personnel  characteristics  as 
represented  by  scores  from  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB). 
Attempts  to  model  these  internal  moderators  have  only  recently  appeared. 
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Personality 

The  study  of  personality  has  long  recognized  the  multi  dimensionality  of  human 
behavior.  For  example,  considerable  recent  work  has  centered  on  the  Five  Factor  Model 
(FFM)  (Barrick,  Mount,  &  Judge,  2001)  as  an  organizing  taxonomy  for  personality.  The 
FFM  consists  of  five  dimensions:  extraversion,  agreeableness,  emotional  stability, 
conscientiousness,  and  openness  to  experience.  Conscientiousness  and  emotional 
stability  have  demonstrated  positive  correlations  with  performance  across  a  wide  variety 
of  jobs  (Barrick  &  Mount,  1991;  McHenry,  Hough,  Toquam,  Hanson,  &  Ashworth,  1990). 
However,  in  general,  job  performance  is  operationalized  as  a  global  outcome  rather  than 
as  discrete  task  performance. 

As  noted,  there  is  evidence  that  personality  impacts  work  performance;  how  will  HSI 
account  for  it  in  models  and  tools?  Numerous  obstacles  to  incorporating  aspects  of 
personality  into  models  of  human  behavior  exist.  Primarily,  most  models  of  personality 
are  not  instantiated  in  computer  models.  There  is  little  research  linking  the  specificity  of 
task  performance  with  explicit  personality  measures  needed  to  improve  human 
performance  models.  For  example,  how  does  the  personality  trait  of  stress  tolerance 
impact  performance  outcomes  of  personnel  involved  in  visual  recognition  tasks?  The 
personnel  domain,  increasingly  engaged  with  personality  constructs,  is  an  excellent 
place  to  look  for  answers. 

The  Navy  Computer  Adaptive  Personality  Scales  (NCAPS)  (Houston,  J.  S.,  &  Borman, 
W.  C.  (2005),  Houston,  J.  S.,  Schneider,  R.  J.,  Ferstl,  K.  L.,  Borman,  W.  C.,  Hedge,  J.  W., 
Farmer,  W.  L.,  &  Bearden,  R.  M.  (2003))  is  a  recently  developed  personality 
measurement  instrument  that  is  part  of  the  larger  Navy  Whole  Person  Assessment 
(WPA)  research  effort.  It  is  a  prime  starting  point  for  the  personality  construct  used  in 
this  effort.  NCAPS  is  an  innovative  instrument  developed  to  assess  personal  attributes 
for  classifying  or  screening  individuals  into  U.S.  Navy  occupations.  The  ASVAB,  a 
collection  of  verbal,  mathematical,  and  technical  tests,  serves  as  the  primary  selection 
and  classification  instrument  for  all  Navy  enlisted  jobs  and  is  a  substantial  predictor  of 
training  outcomes.  However,  many  studies  have  found  that  considering  non-cognitive 
factors  (e.g.,  personality  and  interests)  enhances  the  ability  to  predict  successful 
continuation  and  performance  across  both  civilian  and  military  jobs.  For  example, 
adding  measures  of  conscientiousness  and  emotional  stability  to  cognitive  measures 
such  as  the  ASVAB  can  explain  an  additional  18-38%  of  the  variance  in  job  performance 
(Ferstl,  K.  L.,  Schneider,  R.  J.,  Hedge,  J.  W.,  Houston,  J.  S.,  Borman,  W.  C.,  &  Farmer, 

W.  L.  (2003). 

Stress  tolerance  is  the  personality  construct  that  was  selected  from  the  NCAPS 
battery  to  explore  for  appropriate  inclusion  with  IMPRINT.  The  construct  of  stress 
tolerance  was  selected  because  it  has  been  shown  to  impact  task  performance  and  it  is 
currently  feasible  to  model  the  impact  of  physical  stressors  in  IMPRINT  as  they  apply  to 
tasks  of  certain  types.  The  operational  end  points  of  stress  tolerance  are  defined  as: 

•  High  Scorers— Maintain  composure  and  retain  ability  to  think  clearly  and  take 
effective  action  when  confronted  with  stressful  situations;  can  readily  put  aside 
worries  and  feelings  of  guilt;  can  accept  criticism  without  becoming  upset. 
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•  Low  Scorers— Become  indecisive  or  make  poor  decisions  in  times  of  stress  due 
to  loss  of  composure;  prone  to  feelings  of  worry,  guilt,  and  vulnerability;  are 
easily  upset;  tend  to  ruminate  about  troubling  events  and  perceived  failures;  do 
not  take  criticism  well. 


Computer  Simulations  of  Personality 

There  have  been  a  few  recent  efforts  to  include  personality  in  computer  simulations 
of  human  performance.  McKenzie,  Petty,  and  Catanzaro  (2003)  constructed  a 
simulation  that  uses  a  trait-based  model  of  personality  to  influence  the  decision 
outcomes  of  a  military  commander.  The  simulation  included  anxiety,  anger,  and 
independence.  Oren  and  Ghasem-Aghaee  (2003)  used  the  FFM  as  the  framework  for 
using  fuzzy  logic  to  process  personality  variables  but  did  not  instantiate  the  proposed 
system  into  a  computer  simulation.  Aykroyd,  Bachman,  Harper,  and  Hudlicka  (2005), 
building  on  previous  work  done  with  the  Methodology  for  Analysis  and  Modeling  of 
Individual  Differences  (MAMID)  architecture  used  an  indirect  approach  to  modeling 
personality  traits.  They  argued  that  traits  impact  behavior  influencing  such  cognitive 
attributes  as  attention  and  working  memory,  leading  to  defining  a  cognitive  architecture 
before  modeling  such  individual  differences  as  personality.  Zachary,  LeMentec,  Miller, 
Read,  and  Thomas-Meyers  (2005)  describes  the  Personality-based  Architecture  for 
Cognition  (PAC),  arguing  that  personality  traits  and  cultural  characteristics  are  not 
optional  aspects  of  human  behavior  but  are  part  of  the  foundation  of  human  behavior. 
PAC  uses  personality  traits  as  part  of  the  foundation  for  the  cognitive  process.  Using 
research  results  from  neuroscience  and  research  from  personality  trait  investigation, 
PAC  is  conceptualized  as  a  3-level  representation.  Later  we  will  explain  how  some  of  this 
work  may  be  expanded  upon  to  roll  personality  into  IMPRINT  models  via  external 
connections. 


I  mproved  Performance  Research  I  integration  Tool 


Why  I  MPRI  NT? 

Although  there  are  many  human  performance  analysis  software  tools  currently 
available,  we  chose  IMPRINT  to  present  the  framework  for  modeling  the  effect  of 
personality  on  human  performance  due  to  a  congressional  mandate  contained  in  the 
National  Defense  Authorization  Acts  of  2003  and  2004,  specifying  the  use  of  IMPRINT 
in  the  Navy’s  HSI  program.  In  2003,  following  the  lead  of  the  Army’s  Manpower  and 
Personnel  Integration  (MANPRINT)  program,  the  Navy  initiated  the  HSI  program  to 
define  a  human-centered  design  policy  for  materiel  acquisition.  HSI  formally  requires 
the  implementation  of  the  DoD’s  HSI  policy  for  Navy  materiel  acquisition,  ensuring  the 
systems  engineering  process  includes  the  Sailor  as  an  integral  component  during  design. 
We  also  chose  IMPRINT  because  much  of  the  framework  and  many  of  the  hooks  that 
will  eventually  be  required  to  add  personality  traits  into  human  behavioral 
representations  are  already  in  place  within  IMPRINT.  For  example,  IMPRINT  already 
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has  descriptors  of  tasks  that  allow  the  matching  of  relevant  research  to  performance 
degradations  in  terms  of  task  types.  Further,  algorithms  already  exist  within  IMPRINT 
for  performance  moderators  broken  into  Personnel,  Training  and  Stressors  (PTS),  and 
the  addition  of  personality  constructs  would  be  a  logical  enhancement  to  the  “P”  part  of 
that  equation. 


History  of  I MPRI  NT 

In  the  1970s  the  U.S.  Air  Force,  Navy,  and  Army  recognized  the  need  to  estimate 
manpower,  personnel,  and  training  (MPT)  requirements  early  in  the  system  acquisition 
process.  The  U.S.  Navy  took  the  lead  by  developing  the  HARDMAN  Comparability 
Methodology  (HCM)  to  analyze  the  trade  space  between  hardware  and  manpower. 
Subsequently,  the  U.S.  Army  then  adapted  HCM,  renamed  HARDMAN  I  to  include  a 
broader  range  of  weapon  systems.  A  subsequent  evolution  by  the  U.S.  Army  automated 
the  process  and  was  called  HARDMAN  II.  In  the  mid  to  late  1980s  HARDMAN  II 
evolved,  linking  MPT  to  Warfighter-system  performance  through  a  set  of  innovative 
software  analysis  modules  (Kaplan,  1988).  The  addition  of  the  software  modules 
warranted  a  new  title,  HARDMAN  III.  In  the  1990s,  the  U.S.  Army  ported  HARDMAN 
III  to  the  Microsoft  Windows  platform  and  renamed  the  system  to  the  Improved 
Performance  Research  Integration  Tool  (IMPRINT).  Since  becoming  IMPRINT  the  title 
has  essentially  remained  the  same  with  only  the  version  number  changing  as 
functionality  has  been  improved  and  features  added.  Figure  2  shows  the  history  behind 
the  development  of  IMPRINT. 


1970 

1980 

1990 


Concept  Paper 
~  Air  Force  ~ 

MPT  data  provided 
-  Paper  &  pencil  - 

Automated  process 

-  Mini-computer  - 

MPT  link  to  performance 
-  PC  - 

Integrated  analysis  environment 

-  Windows  - 

Goal  Oriented  Behaviors  &  HLA 
Compliance 


.  Navy  HARDMAN 

(Hardware  vs.  Manpower) 

Army  HARDMAN  II 

*  Army  HARDMAN  III 

*  IMPRINT  &  WinCrew 
. IMPRINT  6 


New  output  reports  &  32  bit  application  ••••*■  IMPRINT  7 

Tri-Service  analysis  capabilities,  New 
interface  &  improved  plug-in  and  links  to  IMPRINT  Pro 
other  simulations 


Figure  2.  The  evolution  of  I  MPRI  NT. 
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MPRI  NT 


IMPRINT  is  a  dynamic,  stochastic,  discrete  event  network  modeling  tool  specifically 
designed  for  representing  and  analyzing  Warfighter-system  performance.  To  analyze 
Warfighter-system  performance,  the  user  must  represent  how  the  total  system,  whether 
hardware,  software,  or  system,  will  be  used  to  accomplish  missions  through  the 
decomposition  of  system  missions  into  discrete  tasks.  Tasks  contain  precise  information 
about  the  mission  such  as  mean  time  to  complete,  mean  accuracy  when  performing, 
assigned  operator  (or  hardware/ software),  and  required  mental  workload  (if  performed 
by  a  human).  Given  that  a  Warfighter-system  can  be  adequately  represented  through  a 
decomposition  of  the  system  mission  into  discrete  tasks,  a  user  can  employ  IMPRINT  to 
answer  a  multitude  of  questions  concerning  the  Warfighter-system.  The  questions  listed 
below  present  some  of  the  questions  an  IMPRINT  user  can  answer: 

•  How  much  mental  workload  must  an  operator  expend  to  perform  a  task? 

•  What  is  the  amount  of  mental  workload  an  operator  will  expend  throughout  the 
mission? 

•  Are  any  operators  mentally  overloaded? 

•  Did  the  actual  mission  time,  as  predicted  by  running  a  simulation,  meet  the 
required  mission  time? 

•  Can  the  mission  be  performed  safely  with  one  less  operator? 


I  MPRI  NT  Performance  Moderators 

In  addition  to  basic  task  network  simulation,  IMPRINT  is  able  to  predict  Warfighter- 
system  performance  under  diverse  conditions  through  the  use  of  its  performance 
moderators’  module  that  consists  of  personnel,  training,  and  environmental  stressor 
moderators.  With  this  feature,  an  analyst  can  understand  the  constraints  imposed  on 
the  system  when  operating  in  adverse  conditions.  For  example,  the  environmental 
component  will  moderate  the  Warfighter  performance  to  account  for  extreme  operating 
conditions.  Currently,  IMPRINT  has  five  environmental  stressors:  heat,  cold,  noise, 
hours  since  waking  (a  cumulative  fatigue  stressor),  and  protective  clothing  (Mission- 
Oriented  Protective  Posture  or  MOPP  gear).  Generally,  the  application  of  a  stressor  will 
in  some  manner  reduce  task  accuracy  and/or  increase  the  time  to  complete  a  task. 

Similar  to  the  environmental  stressors,  IMPRINT  includes  the  performance 
moderators,  personnel  characteristics  and  training  frequency.  The  personnel 
characteristics  moderator  uses  Warfighter  aptitude  to  modify  performance.  Warfighters 
with  higher  aptitude  will  execute  tasks  with  increased  accuracy  and  in  shorter  time 
periods,  while  Warfighters  with  lower  aptitudes  may  perform  with  lower  accuracy  or 
require  more  time  to  perform  the  tasks.  The  training  frequency  moderator  adjusts 
performance  straightforwardly,  improving  task  performance  with  higher  training 
frequencies  and  degrading  performance  when  training  frequency  is  lowered. 
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An  important  clarification  is  that  an  applied  stressor,  whether  environmental, 
personnel,  or  training,  does  not  necessarily  affect  all  the  tasks  of  a  model.  Some  tasks 
remain  unaffected  regardless  of  stressor  application.  Additionally,  stressors  do  not 
uniformly  adjust  time  or  accuracy  for  the  tasks  they  do  affect  in  a  model  (Allender, 
2000).  Consider  the  example  of  a  typist  transcribing  a  report  who  occasionally  needs  to 
converse  in  a  room  with  a  loud  window  air  conditioner.  The  typing  task  will  generally  be 
unaffected  whereas  the  conversing  task  will  be  severely  affected  by  the  noisy  air 
conditioner. 

To  accommodate  these  nuances,  IMPRINT  uses  a  categorical  weighting  scheme  to 
describe  the  required  work  for  performing  a  task  (Fleishman  &  Quaintance,  1984). 

While  Fleishman  identified  53  categories,  IMPRINT  has  winnowed  this  down  to  9 
categories,  referred  to  as  “taxa”  to  describe  a  task  (see  Figure  3).  The  nine  taxa  each 
belong  to  one  of  four  general  descriptor  categories:  Perceptual,  Cognitive,  Motor,  and 
Communication.  One  caveat  of  the  taxon  interface  imposed  on  the  user  is  that  a 
maximum  of  three  taxa  may  be  assigned  to  a  task.  This  three  taxa  limit  originated  due  to 
the  technical  limitations  of  a  third-party  database  used  in  earlier  versions  of  the 
software.  No  change  has  been  made  to  increase  the  three  taxa  maximum  in  subsequent 
releases  of  IMPRINT  because  from  a  practical  perspective  no  significant  difference  in 
performance  would  be  realized  as  a  result  of  applying  a  fourth  stressor. 


Figure  3. 1 MPRI  NT's  nine  taxa. 
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Using  the  taxon  interface  to  describe  the  required  work  of  a  task  allows  the  stressor 
algorithms  to  affect  the  appropriate  tasks  in  the  simulation.  In  the  example  of  the  typist 
discussed  previously,  the  taxon  “Motor:  Fine  Motor— Discrete”  describes  the  typing  task 
and  the  taxon  “Communication:  Oral”  describes  the  conversing  task.  When  the  user 
applies  the  noise  stressor,  the  conversing  task’s  performance  degrades  whereas  the 
typing  task  remains  unchanged.  If,  on  the  other  hand,  the  user  applies  the  cold  stressor 
instead  of  noise,  the  typing  task’s  performance  degrades  and  the  conversing  task’s 
performance  remains  unchanged.  Table  l  reveals  how  the  five  environmental  stressors 
currently  affect  task  performance  through  taxon  assignment. 


Table  1 

Environmental  stressor  effects  on  taxa 


Taxon 

MOPP 

Heat 

Sleepless 

Cold  Noise  Hours 

Visual 

T 

A 

T 

Numerical 

A 

T,  A 

Cognitive 

A 

T,  A 

Fine  Motor  Discrete 

T 

A 

T 

Fine  Motor  Continuous 

Gross  Motor  Light 

T 

T 

Gross  Motor  Heavy 

Commo.  (Read  &  Write) 

A 

Commo.  (Oral) 

T 

A 

A 

Where  T  =  affects  task  time;  A  =  affects  task  accuracy. 


New  Performance  Moderator  I  mplementation  Methods  in  I MPRI  NT 

There  are  four  basic  options  currently  available  for  implementing  new  performance 
moderators  into  IMPRINT.  Each  of  them  will  be  explained  below,  and  a  discussion  of 
potential  “fits”  for  adding  a  personality  construct  moderator,  such  as  stress  tolerance, 
will  be  discussed  in  more  detail  in  the  section  that  follows. 

The  first  option  for  implementing  a  new  performance  moderator  into  IMPRINT  is  to 
add  one  to  the  existing  PTS  interfaces  with  underlying  algorithms  that  impact  task 
performance,  in  terms  of  time  and  accuracy,  based  on  task  types  as  defined  by  taxon 
weights.  For  this  implementation  option,  data  must  be  available  or  obtained  through 
rigorous  studies  on  task  performance  under  varied  conditions  for  the  performance 
moderator  chosen.  This  option  requires  changes  to  IMPRINT  source  code,  and  therefore 
also  supplies  other  IMPRINT  users  with  the  new  functionality. 

The  second  implementation  option  would  be  to  add  a  “custom  stressor”  into  a 
specific  IMPRINT  mission  model  or  collection  of  mission  models  called  an  analysis.  This 
custom  stressor  would  be  hooked  to  task  taxon  weights  similar  to  the  built-in 
moderators;  however,  the  user  will  control  how  tasks  are  changed  based  on  these 
weights  by  supplying  their  own  algorithms  which  may  apply  only  to  a  limited  study.  This 
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implementation  option  does  not  require  any  change  to  IMPRINT  source  code;  and  is, 
therefore,  also  not  reusable  by  other  members  of  the  IMPRINT  user  community  without 
having  to  recreate  it  separately. 

A  third  implementation  option  is  to  use  IMPRINT  user  code,  such  as  variables  and 
expressions  in  release  conditions,  beginning  and  ending  effects,  and  path  logic,  to 
change  the  way  specific  mission  models  are  performed  based  on  “moderator  variables” 
or  conditions  set  by  variables  representing  the  moderators  that  a  skilled  modeler  wants 
to  represent.  This  implementation  method  also  does  not  require  any  change  to 
IMPRINT  source  code,  although  it  is  even  less  transferable  to  other  users  because  it  will 
likely  be  very  specific  to  the  systems  and  missions  being  studied  by  the  modeler. 

A  final  option  for  implementing  performance  moderator  effects  is  to  build  calls  into  a 
model,  and  necessary  software  code  around  a  model,  to  allow  it  to  ask  external 
representations  of  higher  fidelity  “thinkers”  for  behavior.  In  this  option,  the  modeler 
must  also  have  a  representation  of  the  moderators  to  be  used  in  their  own  variables  (as 
in  the  third  option);  however,  if  the  software  around  the  model  (or  around  IMPRINT,  in 
something  called  a  “plug-in”)  is  built  correctly,  there  may  be  some  transferability  and 
reusability  to  this  solution. 


Personality  Construct  Performance  Moderator  Design 

As  we  studied  the  data  and  literature  currently  available  on  personality  constructs 
such  as  those  included  in  NCAPS,  and  then  focused  more  closely  on  the  construct  of 
stress  tolerance,  it  became  apparent  that  currently  available  data  did  not  support  the 
inclusion  of  a  new  personality-based  performance  moderator  into  IMPRINT  per  the  first 
option  listed  in  the  previous  section.  However,  through  research  and  various  discussions, 
we  were  able  to  develop  potential  approaches  for  future  performance  moderator  design, 
should  empirical  data  to  support  the  moderators  become  available. 

Empirically,  it  would  seem  to  make  sense  that  tasks  that  are  largely  cognitive  in 
nature  (in  terms  of  IMPRINT  taxa)  would  be  more  prone  to  be  impacted  by  stress 
tolerance.  However,  just  because  a  task  is  largely  cognitive,  that  does  not  necessarily 
indicate  that  the  task  is  mentally  taxing.  In  fact,  mental  workload  as  represented  by 
IMPRINT  includes  the  mental  capacity  required  to  handle  other  tasks  types  as  well  as 
“cognitive”  tasks,  including  visual  tasks,  motor  tasks,  auditory  tasks,  and  speech  tasks. 

In  addition  to  the  ability  to  represent  tasks  in  terms  of  the  taxa  described  above, 
IMPRINT  also  allows  a  user  to  assign  levels  of  mental  workload  for  each  task  in  different 
mental  workload  channels.  Further,  as  a  model  executes,  total  mental  workload  over 
time  is  calculated,  including  workload  encountered  when  multiple  tasks  are  being 
performed  concurrently. 

We  see  a  potential  opportunity  to  utilize  this  total  mental  workload  encountered, 
potentially  in  combination  with  other  user  defined  variables  related  to  stress  tolerance, 
to  model  both  perceived  levels  of  stress  and  the  impacts  of  those  levels  on  operators  with 
different  levels  of  stress  tolerance.  For  example,  if  an  operator  has  a  high  current  overall 
workload  score,  and  also  is  performing  a  task  that  has  been  tagged  as  “urgent”  or 
“sensitive”  or  “stressful,”  that  operator  may  be  more  likely  to  make  an  error  (which 
could  be  represented  either  via  a  change  in  task  accuracy  or  some  other  variable  that 
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creates  a  “recovery  path”  in  the  model)  or  have  degraded  task  performance.  This 
solution  will  largely  be  based  on  the  third  generic  option  listed  above.  However,  if  data 
to  support  links  between  stress  tolerance,  mental  workload,  and  task  performance  can 
be  found,  some  of  the  individual  algorithms  built  into  test  mission  models  may  be 
transferable  to  other  modelers  and  added  to  IMPRINT. 

Finally,  another  future  implementation  option  or  design  would  be  to  add  personality 
models,  possibly  per  some  of  those  discussed  earlier  in  this  paper,  to  higher  fidelity 
thinking  models  such  as  Recognition  Primed  Decision-making  (RPD)  or  ACT-R. 
IMPRINT  has  already  been  successfully  connected  with  these  higher  fidelity  thinking 
models,  therefore  adding  the  personality  models  to  them  would  allow  their  inclusion  in 
IMPRINT  simulations  should  similar  linkage  occur. 

Future  Personality  Construct  Performance  Moderator  Development 

While  personality  has  been  demonstrated  to  be  a  good  predictor  of  job  performance, 
the  relationship  between  personality  and  discrete  task  performance  in  varying 
environmental  conditions  has  not  been  rigorously  studied.  Nor  has  the  relationship 
between  personality  traits,  work  outcome,  and  the  interplay  of  personality  traits 
between  team  members  been  rigorously  studied.  The  proposed  work  outlined  here  will 
address  the  dearth  of  research  and  may  provide  valuable  information  for  HSI 
developments.  This  work  consists  of  three  phases,  outlined  below. 

Phase  1:  Literature  Review  and  Feasibility  Study 

A  comprehensive  review  of  the  literature  reporting  the  relationships  between 
personality  measures  and  individual  work  outcome,  and  personality  measures  and  team 
performance  will  be  conducted.  An  analysis  of  the  research  to  date  summarizing  past 
research,  emphasizing  discrete  task  performance  for  individuals  and  teams,  and  a 
detailed  plan  for  phase  2  will  be  developed. 


Phase  2:  Proof- of- Concept  Study 

The  performance  measures  of  three  high  fidelity  simulators  measuring  individual 
and  team  performance  and  approved  by  the  HSI  team  will  be  utilized  as  the  criterion 
measures  to  determine  the  relationship  between  individual  and  team  task  performance 
and  relevant  personality  traits  mesasured  by  NCAPS.  Criteria  will  be  established  for 
simulator  selection  which  will  include  but  not  be  limited  to  (1)  high  fidelity 
measurements  of  performance  (reaction  time,  error  rate...),  (2)  realistic  representation 
of  a  variety  of  environmental  conditions,  and  (3)  measurement  of  team  performance. 
The  results  will  quantify  the  relationship  between  personality  levels  and  discrete  task 
performance  for  individuals  and  teams.  The  relationships  between  personality  trait 
levels  required  by  members  of  the  teams  will  also  be  provided.  It  is  possible  that  trait 
level  tradeoffs  can  exist  between  team  members  (i.e.,  one  team  member  with  a 
particular  level  of  a  trait  can  compensate  for  another  team  member  with  a  low  level  of  a 
particular  trait)  and  team  performance  can  remain  optimal. 
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Phase  3:  Live  Operations  Study 


As  part  of  a  live  operations  study,  the  measured  work  performance  of  individuals 
and/or  teams  in  at  least  3  different  work  environments  will  be  utilized  as  the  criterion 
measures  to  determine  the  relationship  between  individual  and  team  task  performance 
and  the  NCAPS  traits.  Trait  levels,  work  outcome,  and  tradeoffs  will  be  examined  and 
recommendations  for  including  personality  into  the  HSI  process  will  be  provided. 


Other  Future  Directions 


In  addition  to  potential  future  work  related  to  WPA  personality  constructs  such  as 
stress  tolerance  to  enhance  personnel  performance  moderators  in  IMPRINT,  the 
authors  also  identified  other  potential  enhancements  that  could  be  made  in  IMPRINT 
applicable  to  the  Navy’s  personnel  and  manpower  domains.  These  ideas  are  briefly 
summarized  below. 


I  nventory  Management  Projections 

IMPRINT  currently  has  Army  specific  personnel  data  libraries  (based  on  Army 
Military  Occupational  Specialty,  or  MOS,  categories)  with  which  it  was  developed  and  a 
primitive  projection  capability  for  looking  at  future  personnel  inventories.  Any  Navy 
IMPRINT-like  HPM  tool  must  be  linked  to  a  more  sophisticated  inventory  management 
tool  that  includes  the  ability  to  model  various  constraints  on  force  levels.  For  example, 
the  Navy’s  Skilled  Personnel  Projection  for  Enlisted  Retention  (SKIPPER,  also  known  as 
Enlisted  Strategic  Analysis  Model  [ESAM])  tool  allows  users  to  project  personnel 
inventory  levels  by  skill  by  length  of  service  (LOS)  and  perform  various  what- if  scenarios 
modeling  the  impact  of  varying  selective  reenlistment  bonuses,  “A”  School  constraints, 
and  other  policy  impacts.  The  costs  of  these  scenarios  could  then  be  modeled  and 
analyzed  as  noted  previously. 


Cost  Range  of  Different  Configurations 

For  a  human  performance  modeling  tool  to  be  of  maximum  utility  for  HSI  within  the 
Navy’s  acquisition  process  it  needs  to  be  closely  coupled  to  a  cost  model.  Information 
about  system  performance  is  valuable;  but  costing  different  configurations  adds 
incredible  utility  and  breadth  to  the  model  and  to  the  acquisition  process.  Adding  cost 
modeling  could  be  done  modularly  so  that  the  model  would  take  input  either  directly 
from  the  HPM  tool  or  from  an  inventory  management  tool  that  projects  over  some  part 
of  the  system’s  lifecycle. 
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Research  Platform 


The  Navy  should  investigate  the  feasibility  of  developing  an  HBR  research  platform 
that  could  be  used  for  advancing  the  “state  of  the  art”  in  HBR.  This  could  be  done 
through  a  consortium  of  universities  and  private  organizations  in  cooperation  with  the 
Navy’s  HSI  office.  Many  of  the  applications  from  this  work  would  contribute  to  the 
Navy’s  training  communities  through  the  development  of  more  realistic  HBR  models, 
including  the  development  of  realistic  opponent  models  used  in  training  simulations. 


HBR  Tools  Do  Not  Specifically  Address  Team  I  ssues 

Current  tools  used  to  model  human  behavior  focus  on  individual  behavior.  Tools 
such  as  IMPRINT,  when  used  by  an  experienced  analyst,  can  show  performance 
conflicts/bottlenecks  that  develop  in  team  situations  but  the  tools  have  been  developed 
with  an  individual  focus.  Research  into  how  team  performance  concepts  can  be  better 
integrated  into  existing  and  future  tools  and  needs  to  be  performed. 
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